This invention relates to computers, and more particularly to improved methods and arrangements for use in generating, encoding, storing, transporting, accessing, and rendering images and animations using image-based rendering (IBR) data.
There is an on-going effort in computer graphics and image-based rendering to provide photorealistic image rendering of scenes. In particular, developers have been searching for methods and arrangements that can provide photorealistic, real-time rendering of dynamically changing scenes. Such scenes include, for example, interactive computer-generated animations, scene walkthroughs, etc.
Unfortunately, the requisite computation needed for such real-time renderings is substantially beyond the capability of most conventional personal computers and workstations, for all but the simplest of scenes.
Conventional personal computers (PCs), workstations, and the like usually have dedicated graphics hardware that is capable of rendering texture-mapped polygons in an efficient manner. In a move to provide more timely image rendering, several image based rendering techniques have been developed that take advantage of this texture mapping capability. By way of example, a view-dependent textures (VDT) method has been presented, in which geometric objects are texture mapped using projective mapping from view-based images.
This and other recent methods, however, usually fail to properly render highly specular surfaces, and often still require high levels of computation and data.
Thus, there is a continuing need for improved methods and arrangements that allow for real-time rendering of scenes having various light sources and objects having differing specular surfaces. Preferably, the methods and arrangements will use conventional graphics hardware configurations, support multidimensional animations, and reduce the amount of data required to render the scene.
Methods and arrangements are provided for real-time rendering of scenes having various light sources and objects having different specular surfaces. The methods and arrangements take advantage of conventional graphics hardware to render texture-mapped polygons, but do so in a view-independent manner.
For example, the above stated needs and other are met by an arrangement that includes an offline encoder employed to parameterize images by two or more arbitrary variables allowing view, lighting, and object changes. The parameterized images can be encoded as a set of per-object parameterized textures based on shading models, viewpoint parameters, and the scene""s geometry. Texture maps are inferred from the segmented imagery of an offline renderer (such as a ray-tracer) to provide the best match when applied to a specific graphics hardware/software rendering arrangement.
In certain implementations, the parameterized textures are encoded as a multidimensional Laplacian pyramid on fixed size blocks of parameter space. This technique captures the coherence associated with objects in the parameterized animations and decodes directly into texture maps that are easy to load into conventional graphics hardware. Certain implementations apply adaptive dimension splitting in the Laplacian pyramid to take advantage of differences in coherence across different parameter dimensions and separate diffuse and specular lighting layers to further improve the compression of the data. As a result of these various methods and arrangements, data compression ratios of greater than about 200:1 can be achieved. Indeed, for example, experiments have shown that data compression ratios as high as about 800:1 can be successfully achieved for real-time interactive play back of animated scenes using conventional graphics cards.